BUG: DataFrame repr with ArrowDtype with extension #54130
Conversation
mroeschke
added
Output-Formatting
| dtype=ArrowDtype(ArrowPeriodType("D")), | ||
| ) | ||
| result = repr(df) | ||
| expected = " col\n0 15340\n1 15341\n2 15342" |
There was a problem hiding this comment.
where does "15340" come from? i'd expect something like "2012-01-01"
There was a problem hiding this comment.
I think this is because ArrowPeriodType stores period data as pa.int64. Since pyarrow.ExtensionType is only used for serialization it appears it doesn't go through period reprs
There was a problem hiding this comment.
im unclear on if this should be considered "wrong"? or just orthogonal to what this PR is doing? seems like this would be an unhelpful repr to give a user
There was a problem hiding this comment.
I think it's a orthogonal incorrect behavior. This PR addresses a bug where pyarrow.ExtensionTypes didn't even show a repr
There was a problem hiding this comment.
maybe a comment that this test shouldn't be interpreted as saying this repr is "correct"?
| else: | ||
| raise NotImplementedError(pa_type) | ||
| elif isinstance(pa_type, pa.ExtensionType): | ||
| return type(self)(pa_type.storage_type).type |
There was a problem hiding this comment.
is this going to match the storage_type?
There was a problem hiding this comment.
Yeah, the python type of the storage_type
|
Might also close #54063? |
I'll leave that open for now. I interpreted that issue as maybe there's a fix that could be backported to 2.0.4 (but I am not sure that there is) |
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.I think it makes sense for the python
.typeof a pyarrow ExtensionType to be its storage's.type.